1. Technical Field
This application relates to managing compression in data storage systems.
2. Description of Related Art
A traditional storage array (which may also be referred to herein as a “disk storage array”, “disk array”, “storage system”, or simply “array”) is a collection of hard disk drives operating together logically as a unified storage device. Storage arrays are designed to store large quantities of data. Storage arrays typically include one or more storage array processors (SPs), for handling both requests for allocation and input/output (I/O) requests. An SP is the controller for and primary interface to the storage array.
Host computers (which may also be referred to herein as “hosts”, “server computers”, or simply “servers”) may access data by sending access requests to the one or more storage arrays. Some storage arrays require that the access requests identify units of data to be accessed using logical volume (“LUN” or “LU”) and block addresses that define where the units of data are stored on the storage array. Such storage arrays are known as “block I/O” storage arrays. In some block I/O storage arrays, the logical volumes presented by the storage array to the host correspond directly to physical storage devices (e.g., disk drives) on the storage array, so that the specification of a logical volume and block address specifies where the data is physically stored within the storage array. In other block I/O storage arrays (referred to as intelligent storage arrays), internal mapping techniques may be employed so that the logical volumes presented by the storage array do not necessarily map in a one-to-one manner to physical storage devices within the storage array. Nevertheless, the specification of a logical volume and a block address used with an intelligent storage array specifies where associated content is logically stored within the storage array, and from the perspective of devices outside of the storage array (e.g., a host) is perceived as specifying where the data is physically stored.
The block I/O storage array keeps track of the logical unit to physical unit associations in a map. The map associates a host logical unit address with a physical device address. The size of the elements in the map is the coarseness of the map. A map that only has a few entries of large extents is a course grain map. A map that has many entries with small extents is a fine grain map. A fine grain map allows more flexibility but generally is too large for all of it to be contained in memory at once. It is possible to use different mapping granularities for different data to achieve a variety of space/performance trade offs.
Performance of a storage array may be characterized typically by the array's total capacity, response time, and throughput. The capacity of a storage array is the maximum total amount of data that can be stored on the array. The response time of an array is the amount of time that it takes to read data from or write data to the array. The throughput of an array is a measure of the amount of data that can be transferred into or out of (i.e., written to or read from) the array over a given period of time.
The administrator of a storage array may desire to operate the array in a manner that maximizes throughput and minimizes response time. In general, performance of a storage array may be constrained by both physical and temporal constraints. Examples of physical constraints include bus occupancy and availability, excessive disk arm movement, and uneven distribution of load across disks. Examples of temporal constraints include bus bandwidth, bus speed, spindle rotational speed, serial versus parallel access to multiple read/write heads, and the size of data transfer buffers.
Thin provisioning is a mechanism that applies to large-scale centralized computer disk storage systems, storage area networks (SANs), and storage virtualization systems. Thin provisioning allows space to be easily allocated to servers, on a just-enough and just-in-time basis. The term thin-provisioning is used in contrast to fat provisioning that refers to traditional allocation methods on storage arrays where large pools of storage capacity are allocated to individual applications, but remain unused.
For example, in a storage consolidation environment, where many applications are sharing access to the same storage array, thin provisioning may allow administrators to maintain a single free space buffer pool to service the data growth requirements of all applications. With thin provisioning, storage capacity utilization efficiency can be automatically increased without heavy administrative overhead. Organizations can purchase less storage capacity up front, defer storage capacity upgrades in line with actual business usage, and save the operating costs associated with keeping unused disk capacity spinning.
Thin provisioning enables over-allocation or over-subscription. Over-allocation or over-subscription is a mechanism that allows server applications to be allocated more storage capacity than has been physically reserved on the storage array itself. This allows flexibility in growth and shrinkage of application storage volumes, without having to predict accurately how much a volume will grow or contract. Physical storage capacity on the array is only dedicated when data is actually written by the application, not when the storage volume is initially allocated.
The thin provisioning technology reduces the waste of storage capacity by preventing allocation of storage capacity to an unwritten data area.
Data processed by a computer such as a server typically has various levels of information content. Even within a single file, many data types and formats may be utilized. Each data representation has a specific meaning and each may hold differing quantities of information. Within the current art, computers process data in a native, uncompressed, format. Thus compressed data must often be decompressed prior to performing various data processing functions or operations.
Several solutions exist within the current art including file by file and block structured compressed data management.
In file by file compression, each file is compressed when stored on disk and decompressed when retrieved. For very small files this technique is often adequate, however for larger files the compression and decompression times can be too slow, resulting in inadequate system level performance. One advantage to file by file compression techniques is that they are easy to develop and are compatible with existing file systems.
Block structured disk compression operates by compressing and decompressing fixed block sizes of data. A group of blocks is compressed and decompressed as a unit. Most compression processes achieve a higher level of effectiveness when operating on larger quantities of data. Restated, the larger the quantity of data processed with a uniform information density, the higher the compressions ratio achieved.
Another advantage of block-based compression is that a storage array can perform the compression without server resources being consumed. For example, the array can perform the compression on a LUN while the server is doing other work or while no server is connected to the LUN (e.g., an archive LUN or a template LUN). Block-based compression can also be used with raw blocks of a LUN without a file system.
With respect to compression itself, there are a variety of data compression processes that are currently available. Many compression processes define one or more parameters that can be varied, either dynamically or a-priori, to change the performance characteristics of the process. For example, with a typical dictionary based compression process such as Lempel-Ziv, the size of the dictionary can affect the performance of the process. Indeed, a large dictionary may be employed to yield very good compression ratios but the process may take a long time to execute; this is an example of “slow compression”, which emphasizes a high compression ratio over compression speed. If compression speed is more important than a high compression ratio, the process can be limited by selecting a smaller dictionary, thereby obtaining a much faster compression time, but at the possible cost of a lower compression ratio; this is an example of “fast compression”, which emphasizes compression speed over a high compression ratio. The desired performance of a compression process and the system in which the data compression is employed, will vary depending on the application.
Thus, one challenge in employing data compression for a given application or system is selecting one or more optimal compression processes from the variety of available processes. Indeed, the desired balance between speed and efficiency is typically a significant factor that is considered in determining which process to employ for a given set of data. Processes that compress particularly well usually take longer to execute whereas processes that execute quickly usually do not compress particularly well.